1. Field of the Invention
The present invention relates to the use of statistical representations of traffic flow in a data processing system, such as during the performance of verification tests for a design of the data processing system.
2. Description of the Prior Art
The ability to effectively and efficiently test and/or validate designs is becoming increasingly important. Typical data processing system designs are rapidly increasing in complexity and furthermore are including circuit blocks designed by a variety of different sources or companies. So called System-on-Chip (SoC) designs that integrate a large number of components on a single integrated circuit have strong advantages in terms of cost and performance, but require significant amounts of validation and testing before the designs can be reliably released for manufacture. Such validation and testing may be performed in respect of the entire design of the data processing system, or in relation to one or more particular components intended for use within the data processing system, for example particular master devices, particular slave devices, interconnect components, etc.
This validation and testing requirement is becoming a bottleneck in getting new systems into the market place. Consequently, measures that can improve the efficiency and effectiveness of such validation and testing of designs are strongly advantageous.
One known approach that can be used to seek to validate the design of a data processing system is to construct a system under verification (SUV) representing at least a part of the design of the data processing system that is to be tested. The SUV can take a variety forms dependent on the verification execution environment in which that SUV is formed. For example the verification execution environment may be implemented using a simulator tool, in which event the one or more components included within the SUV will typically take the form of software models. Alternatively, the verification execution environment may be provided by an emulator, which may for example provide synthesizable hardware on a FPGA (field programmable gate array) platform. In this embodiment, the various components can be viewed as hardware models of the final intended components, but are not the final hardware implementation themselves. In yet another alternative embodiment, the verification execution environment may be provided as an ASIC implementation, in effect providing the system under verification as a full silicon implementation (this latter approach may for example be useful when seeking to perform verification tests on the software running on the hardware platform).
However the SUV is formed, sequences of verification tests can then be performed upon the SUV to cause various transactions to take place between different portions of the system in order to test correct operation (either at the hardware or the software level), each transaction defining one or more transfers between specified portions of the system. Ideally, the behaviour of the SUV should follow as closely as possible the behaviour of the part of the data processing system represented by that SUV. However, this can prove costly both in the time taken to develop the various components to be included in the SUV, and in developing the verification tests to be performed. For example, detailed functional models can be developed for the various components of the SUV which seek to replicate what the real components would do in a real system, and indeed in some SUVs the real components themselves may be used. However, significant development effort is required to produce such functional models, and further, depending on the stage of development during which the verification tests are performed, the actual real components may not be available.
Considering the verification tests themselves, another factor that contributes significantly to the complexity and cost of performing verification tests is the development of suitable transaction sequences to be executed during the verification tests. Detailed test vectors can be written seeking to define specific lists of transactions seeking to replicate the activities that would be observed in the real system, but such detailed lists of vectors take a long time to prepare, and indeed take a long time to execute within the SUV. Further, such text vectors can be difficult to deploy in emulation or ASIC environments due to their large storage requirements.
Accordingly, with the aim of reducing such complexities, another testing approach which has been developed involves the use of transactors to represent one or more components in the SUV, a transactor being a simplified model which can be viewed externally as modelling the same hardware elements as the component(s) which it is substituted for, but which internally is significantly simplified. As an example, whereas a functional model might be developed to represent a particular type of CPU, this can be replaced by a transactor representing a generic master device. Such a transactor can then be arranged to generate random patterns of transfers constrained by a user configuration file, such an approach often being referred to as constrained random testing. One example of a transactor is an eXtensible Verification Component (XVC), and often a number of XVCs may be substituted into the design, with the test actions for separate XVCs being coordinated by a test manager.
Typically, the way in which transfers constituting different transactions are created and handled within the system will be controlled by a transaction protocol. A user configuration file can be used to constrain the random testing so that any transfers issued by the transactor conform to the transaction protocol. Accordingly, the constrained random testing approach can be used to generate a large number of random transfers conforming to the transaction protocol. Further, the transactor can be arranged to execute one or more directed format vectors in order to efficiently verify corner cases (i.e. problems or situations that occur only outside normal operating parameters) in a system design.
By using transactors in the manner described above, this can enable thorough testing of interfaces within the SUV with significantly less complexity and cost than the earlier-mentioned prior techniques employing functional models or detailed test vectors. However, such transactors do not seek to behave in an equivalent way to the components which they have replaced in the design, and accordingly the use of such transactors suffers from a number of drawbacks.
The traffic flow that occurs as a result of constrained random testing using a transactor is not representative of that which will occur between the component(s) represented by the transactor and the SUV in the real system, and accordingly optimisation or performance measuring is not possible. Further, it can be difficult to reach specifically desired coverage points based solely on constrained random testing, due in part to the large protocol space associated with modern transaction protocols such as AXI. Furthermore constraining the random testing in some meaningful way is difficult to achieve in a readily visible manner, and constraints are usually hard-coded into the transactor, thus increasing complexity and cost of the testing procedure.
A number of papers have been written that describe traffic generation driven by a stochastic model (i.e. a model that approximates the behaviour of a real component using random parameters). For example, the article “Traffic Configuration for Evaluating Networks on Chips”, by Z Lu et al, iwsoc, pages 535 to 540, Fifth International Workshop on System-on-Chip for Real-Time Applications (IWSOC'05, 2005), describes a traffic configuration tree used to capture the spatial distribution, temporal characteristics and message size specifications of a single communication channel, such a channel being a logical path from a source node to a destination node. These properties are captured as statistical aggregations. This representation is then used to generate synthetic network traffic based on estimates of a how a processing element may perform.
The article “Analysis and Synthesis of Cycle-Accurate On-Chip Traffic with Long-Range Dependence”, by A. Scherrer et al, Technical Report 2005-53, LIP, ENS-Lyon, Dec. 2005, describes a methodology for extracting statistical attributes on-chip traffic from a VCD (Value Change Dump) waveform (a VCD being a form of activity reference list) in order to drive a synthetic trace generator.
The Verification Methodology Manual for System Verilog (VMM) describes random scenario generation, where each scenario has a hard coded set of weights to use for generation. Different distributions can be maintained for each attribute.
Whilst the various techniques described above provide various approaches for synthetic traffic generation, a problem still exists that when a transactor is used to interface with a system under verification in order to cause such synthetic traffic to be generated, that traffic is still not closely representative of the traffic that would actually occur in the real system and so is not useful for performance evaluation and optimisation.